Human activity recognition by combining discriminative and generative classifiers

In this article, we propose a novel algorithm for the recognition of complex activities in multimedia streams. The algorithm consists of a discriminative feature classifier based on random forests and a generative classifier, for which we use the hierarchical hidden Markov model. The discriminative feature classifier checks the existence or absence of the steps required for the execution of an activity, while the generative classifier encodes the ordering of these steps. The parameters of the classifier are learned automatically from expert labelled data. The classification output is a label indicating, for an input multimedia stream illustrating a complex activity, the type of the activity performed in the stream and whether this activity was performed in a correct or incorrect/anomalous manner. Results for the publicly available bridge design dataset show that our algorithm offers higher accuracy in activity recognition than other leading methods. Keywords—activity recognition, random forest, hierarchical hidden Markov model.